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In re Patent Application of 



New York, New York 



Mika VALKONEN et al 



Date: February 29, 2000 
Group Art Unit : 



Serial No , : 



Filed: 



Examiner : 



For: METHOD FOR SYNCHRONISING IMAGE DATA OBTAINED FROM PROCESS 
MONITORING CAMERAS 



Hon, Commissioner of Patents 

and Trademarks 
Washington, D.C. 2 0231 



Prior to examination, please amend the application as 



Please amend claims 3-5 as follows. 
Claim 3, line 1, delete "or 2". 

Claim 4, line 1, change "any of the claims 1 to 3" to 
--claim 1-- . 

Claim 5, line 1, change "any of the claims 1 to 4" to 
- -claim 1- - . 

Please add the following new claims 6-16. 



--6. A method as claimed in claim 2, characterised in 
that the process is paper manufacture and the object being 
monitored is the paper web running in the paper machine. 



PRELIMINARY AMENDMENT 



Sir: 



follows : 



IN THE CLAIMS: 
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7. A method as claimed in claim 2, characterised in 
that camera-specific image data is analysed and image variation 
data based on the level of variation in a plurality of sequential 
images is compiled, and the image variation graph corresponding 
to the image variation data of images preceding and following the 
image to be analysed is displayed on the screen. 

8. A method as claimed in claim 3, characterised in 
that camera- specif ic image data is analysed and image variation 
data based on the level of variation in a plurality of sequential 
images is compiled, and the image variation graph corresponding 
to the image variation data of images preceding and following the 
image to be analysed is displayed on the screen. 

9. A method as claimed in claim 6, characterised in 
that camera-specific image data is analysed and image variation 
data based on the level of variation in a plurality of sequential 
images is compiled, and the image variation graph corresponding 
to the image variation data of images preceding and following the 
image to be analysed is displayed on the screen. 

10. A method as claimed in claim 2, characterised in 
that the output levels of the image variation data of the 
different camera positions are standardised so as to be mutually 
comparable, the standardised image variation levels of the 
different camera positions are compared, and the image data 

{2d^ - 2dn) of the camera position representing the highest-level 
variation is selected for automatic display. 

11. A method as claimed in claim 3, characterised in 
that the output levels of the image variation data of the 
different camera positions are standardised so as to be mutually 
comparable, the standardised image variation levels of the 
different camera positions are compared, and the image data 
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(2di - 2dJ of the camera position representing the highest-level 
variation is selected for automatic display. 

12, A method as claimed in claim 6, characterised in 
that the output levels of the image variation data of the 
different camera positions are standardised so as to be mutually 
comparable, the standardised image variation levels of the 
different camera positions are compared, and the image data 

(2di - 2dJ of the camera position representing the highest -level 
variation is selected for automatic display. 

13, A method as claimed in claim 4, characterised in 
that the output levels of the image variation data of the 
different camera positions are standardised so as to be mutually 
comparable, the standardised image variation levels of the 
different camera positions are compared, and the image data 

(2di - 2dJ of the camera position representing the highest-level 
variation is selected for automatic display. 

14, A method as claimed in claim 7, characterised in 
that the output levels of the image variation data of the 
different camera positions are standardised so as to be mutually 
comparable, the standardised image variation levels of the 
different camera positions are compared, and the image data 

(2d-L - 2dJ of the camera position representing the highest -level 
variation is selected for automatic display. 

15, A method as claimed in claim 8, characterised in 
that the output levels of the image variation data of the 
different camera positions are standardised so as to be mutually 
comparable, the standardised image variation levels of the 
different camera positions are compared, and the image data 

(2d-L - 2dJ of the camera position representing the highest-level 
variation is selected for automatic display. 



OO443072 .1 



3 



16. A method as claimed in claim 9, characterised in 
that the output levels of the image variation data of the 
different camera positions are standardised so as to be mutually 
comparable, the standardised image variation levels of the 
different camera positions are compared, and the image data 
(2di - 2d^) of the camera position representing the highest -level 
variation is selected for automatic display.-- 

REMARKS 

This Preliminary Amendment is submitted to change the 
multiple dependent claims to single dependent claims in order to 
reduce the government filing fee, 

EXPRESS MAIL CERTIFICATE Respectfully submitted. 



I hereby certify that this correspondence is being 
deposited with the United States Postal Service as 
Express Mail Post Office to Addressee (mail label # 
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Commissioner of Patents and Trademarks, Washington, 
D.C. 2023 L on February 28, 2000: 



Dorothy Jenkins 

Name of Person Mailing Correspondence 
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Method for synchronising image data obtained from process monitoring cameras 

The object of the invention is a method for synchronising image data obtained 
from process monitoring n;5mer^<5, in which method 
5 ^ different positions In the process are imaged using various cameras; 

image dala froin tlie diiTerent camera positions is stored per camera into 

digital image processors; 
- images stored at the different camera positions are selected for display and 

analysis on the operator's computer screen; and 
10 - from tho image data obtained at the different camera positions are searched 

impiga? depicting the same area in the web by using synchronisation means; 
When the operator finds an inmrfisting object among the images of a particular 
camera position, by utilising synchronisation he will be able to see the sam^ 
area in the paper web when it passed other camera positions, This type of 
1 5 synchronisation has long been a characteristic of Qie Applicant's operations. 
However, its use involves certain problems or deficiencies. Due to the high 
speed and stretching of the web, it is not possible for the synchronisation to be 
<?n accurate as to necessarily show the corresponding area in the new position 
which the synchronisation produces. If there is, for example, an interruption at 
20 the dry end of the machine and the operator of the monitoring system searches 
for the sariiy area in the web at the wet end of the machine, the operator cannot 
be absolutely certain whether tlie object is located in that area of the video 
recording displayed as a result of synchronisation, in whicli case the up«rator is 
likely to discontinue the search. 

25 

*l"he ?iim of the present invention is to provide an improved synchronisation 
method, so that the operator will know for certain that he will arrive automatically 
in the middle of the visualised area^ inside v^icJi the desired same area of the 
weo can be fourid. The idea is, therefore, to compel the operator to go through 
30 an area of sequential images of a size that can be paiarnatrised. In such a case, 
the corresponding area in the web in different camera positions will be found 
more reliably and easily. 
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The characteristic features of the method relating to the invention are presented 
in Ihe appended claims 1 and 2. The dependent claims disclose preferred 
embodiments of the invention. 

5 An exemplary embodiment of the invention is described in the following with 
re^erenc^ lo Ll ie accuiiipanying drawings m whicfi 

Figure 1 shows the system used for implementing the synchronisation method 
relating to the invention as o flow chart, and 

10 

Figure 2 illustrates the performance of synchronisation on the basis of image 
data from ditferfint camera positions. 

Tha general structure and upt^ralion uf ttie system are first described before 
1 5 disclosing the manner according to the invention for visualisinp [he are^i of 
sequential images to be synchronised to the operator. 

In the flow chart shown in Figure 1 , the image source is a video camera 1 which 
produces a cnntinunns video image of the object being monitored. The image 

20 data is processed In a digital signal processor or USP processor. Signal 

processur:> are similar to ordinary microprocessors, but contain, among others, 
the use of floating-point number lechnolugy and address formats for easier 
addressing. Furthermore, the DSP is markedly differenl FiOffi Ih^ uiUinary 
processor in terms of its architecture, having been designed for duties involving 

25 large amounts of data, and where multiplication with integers and data transfers 
are statistically significant operations. DSP processors arc capable of 
performing a plurality of different and simultaneous calculation routines 
associated with Image analysis, thf^ rf^sults of which can be applied 
auloniaLically Lu monitoring changes in Image data. 

30 



The system comprises several video cameras 1 for inidyiny Ltie various 
positions of the process being monitored. Each camera is provided with its own 



28/02 ^00 13:01 FAX 3SS 9 8S599210 



LEIT2INGER OY 



f2]004/011 



3 

digital image processor 2 for storing digital image data per camera. The signal 
processors 2 are used for analysing each camera-specific image data item so 
as to provide image variation data based on the level of variation in a plurality of 
fi^^qiienti?5l im^^ges. hrom the signal processors 2 the said image data and image 
5 variation data are transmitted to an analysing nnmputer 16 which has a display 
11. 

The images stored at the different camera positions can be selected for analysis 
by means of selector icons 7. The image variation graph 8 corresponding to the 

1 0 (mage variation data of images preceding and following the image to be 
analysed is displayed at the bottom of the screen 11 . A floating arrow 
designated hy reference numeral 9 indicates the point on the image variation 
graph 8 at which the image 6 displayed Is locaTed 'i ne tinating arrow 9 c^jn be 
used for selecting an irirdye at any point on the graph 8 for display. The Image 

15 data 2di - 26^, stored from each cameia pusiliun 1-n may encompass several 
hundred images. Depending on the process, the image data in store at each 
time may have a duration ranging from a few seconds to several minutes, and 
the storage may function on the FIFO {first in first out) principle. 

♦ 

20 Since each Image variation graph 8 shown at thp hottom of the screen 1 1 is 
prepared uf image material from each camera position respectively, The im?3ge 
variation graphs of the differenl cainefa positions can be standardised so as to 
be comparable, and be compared with each ulfiei. whereby the image variation 
graph representing the highest-level variation and the image data of Lhe camera 

25 position corresponding to it can be selected automatically for displaying on the 
analysing window of the screen 11. For the purpose of this automation, the 
system i?; provided with standardising means 3 for standardising the output 
levels Of the image vari;^tion data of the different camera positions to be 
mutually comparaOle. A companson between the standardised image variation 

30 levels of the different uamer a positions is performed by means of comparator 
means 4. Selector means G receive ijuaye data from the different camera 
positions and select, under the control of the comparator means 4, ll*ie iniaye 
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data 2di - 2dn representing the highest-level variation and the corresponding 
image variation graph 8 to be displayed on the screen 1 1 . image 6 is one of a 
multitude of images included in the image data. The graph 8 and the floating 
indicator 9 can be used for examining those very images S associated with the 
b area ot the highest-level imags variation. The image 6 on the screen 1 1 may 
represent, for example, a hole? formed in p\ paper web. 

The automated seleciiun uf ttie image 6, as described above, is obviously 
optional, meaning that the operator may, if he so dei>iies>, i>e!ec;t image data 
10 2dt - 2<i^ from any camera position for analysis. However, it is often useful for 
the operator to know at the very beginning of analysis which camera position 
produced the highest-level image variation, in which case the analysis can be 
started trom this particular camera position. 

15 The operator's data processing means 15 comprise synchrnmsatmn means 12, 
13, 14 by means of which iiriages depicting the same area In the web can be 
sought automatically from the image data oruirfeierit camera positions. When 
the synchronisation option 12 is displayed on the screen 11, the synchronisation 
unit 13 controls the image data selection unit 6 in such a way that the selection 

20 of a camera position by means of the selector icons 7 automatically produces on 
the screen 1 1 the image 6 which corresponds to the same area in the paper 
web as the image 5 of the previous camera position. For this purpose the 
syndiiurnsation unit 13 requires certain parameters 14, whidi include at least 
web speed and the distances between the cameras 1. The synchronisation unit 

25 13 is also given a time of occurrence 21 al whidi an interesting object was 

obeen/ed in the imogc field of a camera position, the web area corresponding lu 
which having to be analysed from the different camera positions. 

Figure 2 shows a hnie, for example in the size press, in an image produced at 
30 paint of time 2t in the Image data i^'d,, nf namfira n. On the basis of the time of 
occurrence iind ulher parameter data, the synchronisation unit 1H ik able to 
determine, by means of simple calculcjiions, ttie images Zs. - 2S3 In the image 
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data 2cli - Zd^ in which the same hole appears. The accxjracy of synchronisation 
cannot, however, be very high due to the high speed and stretching of the web, 
and thus in the invention is specified a marker 10, 10* of a certain width, which is 
visualised for the operator, within which the corresponding point in the web will 
5 be found with certainty. For the operator is thus visualised a selection arpa 1 0 in 
the environment of the point of synchronisation 9, 9' of each camera position, 
wittiin which area there is a limited number of sequential images among which 
the corresponding point will be found with certainly. The said limited number is 
preferably a minimum number determined by the parameters such as web speed 
10 and distance between the camera positions. This marker area 10 motivates the 
operator to search for the corresponding point in the different camera positions'. 
The size of the selection area 1 0, that is, the number of images contained by it 
is, therefore, dependent on wsh speRri and the distances between the cameras. 
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Claims 

1 . A method for synchronising image data obtained from procxjss monitoring 
cameras, in which method 

5 - different positions in the process are imaged using various cameras (1 ); 

- image data {2d^ - 2d J from The different camera positions is stored per 
camera into digital image processors (2): 

- images stored al Ihe differ ent cjaineia poisitioris are selected for display and 
analysis on the operator's computer (15) screen (11). and 

10 - from the imago data (2d| - 2dr,) obtained at the different camera positions are 
searched (2t, 2$^ - 2S3 ) images depicting the same area in the web by using 
synchronisation means (12, 13, 14), 
characterised in That tnr the operator is visualised a selection area (10) 
corresponding to the limited numhf^r at sequential images in the environment of 
1 5 the point of synchronisation (9, 9*) of each' camera position 

2. A method for synchronising image data obtained froni piuceiiii rnunilonng 
cameras, in which method 

' different positions in the process arc imaged using various cameras (1 ); 
2D - image data (26^ - 2d„) from the different camera positions is stored per 
camera into digital image processors (2); 

- images stored at ths different camera positions are selected for display and 
analysis on the operator's computer (15) screen (11); and 

- from the image data (2d^ - 2d J obtained at the different camera positions are 
25 searched (2t, 23^ - 2S3) images depicting tl'ie same ^[ea in tfie web by using 

synchronisation means (12, 13, 14), 
characterised in that for the operator is visualised a selection area (10» 10') 
representing the limited number of sequential images, the image (23^ - 2S3) 
inside the said area being brought to the screen (1 1 ) by the synchronisation 
30 means when the image on th^a screen moves from one camera position to 
anultiei, Ihe size of the selection area (10). that is, the numh^r nt im^jgp..? 
contained by it, being dependeril un the speed of the paper web being 
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monitored and the distances between the Cameras. 

3. A mothod as claimed in claim 1 or 2, chai^ctdrised in that th© process is 
paper manufacture and the object being m'onitored is the paper web running in 

5 the paper machine. 

4. A method as claimed in any of thfi claims 1 to 3^ characterised in that 
camera-specific image data is analysed and image variation data based on the 
iev^l of variation in a plurality of sequential images is compiled, and the image 

1 0 variation graph (0) corresponding to the image variation data of images 

preceding and following the image to be analysed is displayed on the screen 
{11). 

b- A method as claimed in any of the claims 1 to 4, characterised in that the 
15 output levels ot the image variation data of the different camera positions are 
standardised so as to be mutually comparable, the standardised image variation 
levels of the differerjl ud(ii^#f a positions are compared, and the image data (Zd, - 
2df,) of the camera position representing the highest-level variation is selected 
for automatic display. 
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(57) Abstract 

The object of the invention is a method for synchronising image data obtained 
from process monitoring cameras. By utilising synchronisation, one and the 
same area In a paper weta can he sought as it passed the different camera 
positions. Since the speeds are high and the webs somewhat stretching, 
syi idiiunisalion dues not often dirtsctly result in the same area being stiown in 
the different camera positions. To faclKtate searching, a selection area (1 0) 
corresponding to the limited number of sequential images in the environment of 
the point of synchronisation (9, 9') of each c^mer^ position ie visualised for the 
operator, inside which area the corresponding area in the web can be found with 
certainty 
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